Search for: All records

This paper studies the problem of data-driven combined longitudinal and lateral control of autonomous vehicles (AVs) such that the AV can stay within a safe but minimum distance from its leading vehicle and, at the same time, in the lane. Most of the existing methods for combined longitudinal and lateral control are either model-based or developed by purely data-driven methods such as reinforcement learning. Traditional model-based control approaches are insufficient to address the adaptive optimal control design issue for AVs in dynamically changing environments and subject to model uncertainty. Moreover, the conventional reinforcement learning approaches require a large volume of data, and cannot guarantee the stability of the vehicle. These limitations are addressed by integrating the advanced control theory with reinforcement learning techniques. To be more specific, by utilizing adaptive dynamic programming techniques and using the motion data collected from the vehicles, a policy iteration algorithm is proposed such that the control policy is iteratively optimized in the absence of the precise knowledge of the AV’s dynamical model. Furthermore, the stability of the AV is guaranteed with the control policy generated at each iteration of the algorithm. The efficiency of the proposed approach is validated by SUMO simulation, a microscopic traffic simulation platform, for different traffic scenarios. 

In this paper, we have proposed a resilient reinforcement learning method for discrete-time linear systems with unknown parameters, under denial-of-service (DoS) attacks. The proposed method is based on policy iteration that learns the optimal controller from input-state data amidst DoS attacks. We achieve an upper bound for the DoS duration to ensure closed-loop stability. The resilience of the closed-loop system, when subjected to DoS attacks with the learned controller and an internal model, has been thoroughly examined. The effectiveness of the proposed methodology is demonstrated on an inverted pendulum on a cart. 

The majority of the past research dealing with lane-changing controller design of autonomous vehicles (𝐴𝑉 s) is based on the assumption of full knowledge of the model dynamics of the 𝐴𝑉 and the surrounding vehicles. However, in the real world, this is not a very realistic assumption as accurate dynamic models are difficult to obtain. Also, the dynamic model parameters might change over time due to various factors. Thus, there is a need for a learning-based lane change controller design methodology that can learn the optimal control policy in real time using sensor data. In this paper, we have addressed this need by introducing an optimal learningbased control methodology that can solve the real-time lane-changing problem of 𝐴𝑉 s, where the input-state data of the 𝐴𝑉 is utilized to generate a near-optimal lane-changing controller by approximate/adaptive dynamic programming (ADP) technique. In the case of this type of complex lane-changing maneuver, the lateral dynamics depend on the longitudinal velocity of the vehicle. If the longitudinal velocity is assumed constant, a linear parameter invariant model can be used. However, assuming constant velocity while performing a lane-changing maneuver is not a realistic assumption. This assumption might increase the risk of accidents, especially in the case of lane abortion when the surrounding vehicles are not cooperative. Thus, in this paper, the dynamics of the 𝐴𝑉 are assumed to be a linear parameter-varying system. Thus we have two challenges for the lane-changing controller design: parameter-varying, and unknown dynamics. With the help of both gain scheduling and ADP techniques combined, a learning-based control algorithm that can generate a near-optimal lane-changing controller without having to know the accurate dynamic model of the 𝐴𝑉 is proposed. The inclusion of a gain scheduling approach with ADP makes the controller applicable to non-linear and/or parameter-varying 𝐴𝑉 dynamics. The stability of the learning-based gain scheduling controller has also been rigorously proved. Moreover, a data-driven lane-changing decision-making algorithm is introduced that can make the 𝐴𝑉 perform a lane abortion if safety conditions are violated during a lane change. Finally, the proposed learning-based gain scheduling controller design algorithm and the lane-changing decision-making methodology are numerically validated using MATLAB, SUMO simulations, and the NGSIM dataset. 

This paper introduces a learning-based optimal control strategy enhanced with nonmodel-based state estimation to manage the complexities of lane-changing maneuvers in autonomous vehicles. Traditional approaches often depend on comprehensive system state information, which may not always be accessible or accurate due to dynamic traffic environments and sensor limitations. Our methodology dynamically adapts to these uncertainties and sensor noise by iteratively refining its control policy based on real-time sensor data and reconstructed states. We implemented an experimental setup featuring a scaled vehicle equipped with GPS, IMUs, and cameras, all processed through an Nvidia Jetson AGX Xavier board. This approach is pivotal as it addresses the limitations of simulations, which often fail to capture the complexity of dynamic real-world conditions. The results from real-world experiments demonstrate that our learning-based control system achieves smoother and more consistent lane-changing behavior compared to traditional direct measurement approaches. This paper underscores the effectiveness of integrating Adaptive Dynamic Programming (ADP) with state estimation techniques, as demonstrated through small-scale experiments. These experiments are crucial as they provide a practical validation platform that simulates real-world complexities, representing a significant advancement in the control systems used for autonomous driving. 

In this letter, we present an active learningbased control method for discrete-time linear systems with unknown parameters under denial-of-service (DoS) attacks. For any DoS duration parameter, using switching systems theory and adaptive dynamic programming, an active learning-based control technique is developed. A critical DoS average dwell-time is learned from online inputstate data, guaranteeing stability of the equilibrium point of the closed-loop system in the presence of DoS attacks with average dwell-time greater than or equal to the critical DoS average dwell-time. The effectiveness of the proposed methodology is illustrated via a numerical example. 

In this paper, we solve the optimal output regulation of discrete-time systems without precise knowledge of the system model. Drawing inspiration from reinforcement learning and adaptive dynamic programming, a data-driven solution is developed that enables asymptotic tracking and disturbance rejection. Notably, it is discovered that the proposed approach for discrete-time output regulation differs from the continuous-time approach in terms of the persistent excitation condition required for policy iteration to be unique and convergent. To address this issue, a new persistent excitation condition is introduced to ensure both uniqueness and convergence of the data-driven policy iteration. The efficacy of the proposed methodology is validated by an inverted pendulum on a cart example. 

In this paper, we address the problem of model-free optimal output regulation of discrete-time systems that aims at achieving asymptotic tracking and disturbance rejection when we have no exact knowledge of the system parameters. Insights from reinforcement learning and adaptive dynamic programming are used to solve this problem. An interesting discovery is that the model-free discrete-time output regulation differs from the continuous-time counterpart in terms of the persistent excitation condition required to ensure the uniqueness and convergence of the policy iteration. In this work, it is shown that this persistent excitation condition must be carefully established in order to ensure the uniqueness and convergence properties of the policy iteration. 

This paper presents a learning-based methodology for developing an optimal lane-changing control policy for a Remote Controlled (RC) car using real-time sensor data. The RC car is equipped with sensors including GPS, IMU devices, and a camera integrated in an Nvidia Jetson AGX Xavier board. By a novel Adaptive Dynamic Programming (ADP) algorithm, our RC car is capable of learning the optimal lane-changing strategies based on the real-time processed measurement from the sensors. The experimental outcomes show that our learning-based control algorithm can be effectively implemented, adapt to parameter changes, and complete the lane changing tasks in a short learning time with satisfactory performance.